Designfax weekly eMagazine

Subscribe Today!

Archives

View Archives

Partners

Manufacturing Center
Product Spotlight

Modern Applications News
Metalworking Ideas For
Today's Job Shops

Tooling and Production
Strategies for large
metalworking plants

Army-funded visual-effects technology used in Avatar, The Jungle Book, and Ready Player One wins Oscar

The Light Stage X capture system. [Credit: U.S. Army Institute for Creative Technologies]

 

 

Creative geniuses behind digital humans and human-like characters in Hollywood blockbusters Avatar, Blade Runner 2049, Maleficent, Furious 7, The Jungle Book, Ready Player One, and others have U.S. Army-funded technology to thank for visual effects.

That technology was developed at the U.S. Army Institute for Creative Technologies (ICT) at the University of Southern California (USC). The ICT is funded by the RDECOM Research Laboratory, the Army's corporate research laboratory (ARL).

Developers of that technology were recently announced as winners of one of nine scientific and technical achievements by the Academy of Motion Picture Arts and Sciences.

A Technical Achievement Award will be presented at the Beverly Wilshire in Beverly Hills on Feb. 9, 2019, to Paul Debevec, Tim Hawkins, and Wan-Chun Ma for the invention of the Polarized Spherical Gradient Illumination facial appearance capture method, and to Xueming Yu for the design and engineering of the Light Stage X capture system, during the Academy's annual Scientific and Technical Awards Presentation. The Scientific and Technical Academy Awards demonstrate a proven record of contributing significant value to the process of making motion pictures.

The Light Stage recording process. [Credit: U.S. Army Institute for Creative Technologies]

 

 

The Academy Certificate reads: "Polarized Spherical Gradient Illumination was a breakthrough in facial capture technology allowing shape and reflectance capture of an actor's face with sub-millimeter detail, enabling the faithful recreation of hero character faces. The Light Stage X structure was the foundation for all subsequent innovation and has been the keystone of the method's evolution into a production system."

The new high-resolution facial scanning process uses a custom sphere of computer-controllable LED light sources to illuminate an actor's face with special polarized gradient lighting patterns that allow digital cameras to digitize every detail of every facial expression at a resolution down to a tenth of a millimeter.

A 2011 USC Institute for Creative Technologies abstract on this tech development further explains why it is so special: "Key to our method is a new pair of linearly polarized lighting patterns which enables multiview diffuse-specular separation under a given spherical illumination condition from just two photographs. The patterns -- one following lines of latitude and one following lines of longitude -- allow the use of fixed linear polarizers in front of the cameras, enabling more efficient acquisition of diffuse and specular albedo and normal maps from multiple viewpoints. In a second step, we employ these albedo and normal maps as input to a novel multi-resolution adaptive domain message passing stereo reconstruction algorithm to create high-resolution facial geometry. To do this, we formulate the stereo reconstruction from multiple cameras in a commonly parameterized domain for multiview reconstruction. We show competitive results consisting of high-resolution facial geometry with relightable reflectance maps using five DSLR cameras. Our technique scales well for multiview acquisition without requiring specialized camera systems for sensing multiple polarization states."

The technology has been used by the visual effects industry to help create digital human and human-like characters in a number of movies and has scanned more than 100 actors, including Tom Cruise, Angelina Jolie, Zoe Saldana, Hugh Jackman, Brad Pitt, and Dwayne Johnson, at USC ICT.

Additionally, the Light Stage technology assists the military in facilitating recordings for its Sexual Harassment/Assault Response and Prevention program through a system called The Digital Survivor of Sexual Assault (DS2A). DS2A leverages research technologies previously created for the Department of Defense under the direction of the U.S. Army Research Laboratory and allows for Soldiers to interact with a digital guest speaker and hear their stories. As part of the ongoing SHARP training, this technology enables new SHARP personnel, as well as selected Army leaders, to participate in conversations on SHARP topics through the lens of a survivor's firsthand account. It is the first system of its kind to be used in an Army classroom.

The recording process for DS2A. [Credit: U.S. Army Institute for Creative Technologies]

 

 

All four awardees were members of USC ICT's Graphics Laboratory during the development of the technology from 2006 through 2016.

Paul Debevec [Credit: U.S. Army Institute for Creative Technologies]

 

 

 

 

Paul Debevec continues as an Adjunct Research Professor at USC Viterbi and at the USC ICT Vision & Graphics Lab. Wan-Chun "Alex" Ma was Paul Debevec's first Ph.D student at USC ICT, and Xueming Yu joined the USC ICT Graphics Lab in 2008 as a USC Viterbi Master's student. Tim Hawkins now runs a commercial light stage scanning service in Burbank for OTOY, who licensed the light stage technology through USC Stevens in 2008.

This is the second Academy Sci-Tech award being given to the Light Stage technology developed at the USC Institute for Creative Technologies. The first, given nine years ago, was for the earliest light stage capture devices and the "image-based facial rendering system developed for character relighting in motion pictures" and was awarded to Paul Debevec, Tim Hawkins, John Monos, and Mark Sagar.

Sources: U.S. Army RDECOM Research Laboratory, USC

Published January 2019

Rate this article